Using Lexical Knowledge in Text Classification

This paper describes several experiments in text classification using WordNet, a rich source of lexical background knowledge available in the public domain. WordNet is used to map the original words from a text into sets based on synonym and hypernym relationships. This information is used to compute a change of representation from bag of words to hypernym density. Six binary classification tasks of varying difficulty are defined, and the Ripper system is used to produce discrimination rules for each task using each representation. Experiments show that for some of the more difficult tasks the hypernym density representation leads to significantly more accurate and more comprehensible rules.